Systems and methods for training image processing models for vehicle data collection

ABSTRACT

Systems and methods for training image processing models for vehicle data collection by image analysis are provided. An example method involves accessing an image of a field of interest in a vehicle captured by a camera in the vehicle, providing a user interface to, display the image, receive input that defines a region of interest in the image that is expected to convey vehicle information, and receive input that assigns a label to the region of interest that associates the region of interest with an image processing model that is to be trained to extract a type of vehicle information from the region of interest, and contributing the image, labelled with the region of interest and the label associating the region of interest to the image processing model, to a training data library to train the image processing model.

RELATED APPLICATIONS

This application claims the benefit under 35 U.S.C. § 119(e) to U.S.provisional Application Ser. No. 63/162,295, titled “SYSTEMS AND METHODSFOR VEHICLE DATA COLLECTION BY IMAGE ANALYSIS”, filed on Mar. 17, 2021,which is herein incorporated by reference in its entirety.

FIELD

The present disclosure relates to telematics, and in particular to thecollection of data from vehicles by telematics systems.

BACKGROUND

In the field of telematics, the location of a vehicle and other datapertaining to the vehicle may be monitored by a telematics system. Thetelematics system may be in the form of a device placed onboard thevehicle, or a system integrated within the vehicle itself, incommunication with a remote data collection system. The location of thevehicle may be tracked through the use of a satellite navigation system,such as a Global Positioning System (GPS), Global Navigation SatelliteSystem (GNSS), cellular tower network, or another system. Additionaldata may be collected through sensors (e.g., accelerometers, temperaturesensors), or by decoding data messages received from Electronic ControlUnits (ECUs) of the vehicle. If available, ECUs may provide informationabout the operating status of the vehicle, such as engine speed, batterytemperature, fuel level, tire pressure, odometer reading, and otherdata. Such data may be made available over a Controller Area Network(CAN) bus, through a communication port (e.g., an OBD2 port), and/or bydirect connection to systems onboard the vehicle. In any case, such datamay be transmitted to, and recorded at, a vehicle data collection systemto be used in the provision of a telematics service, such as a fleetmanagement tool, or for further analysis.

SUMMARY

According to an aspect of the disclosure, an example method for vehicledata collection is provided. The method involves positioning a camera ina vehicle to be pointed toward a field of interest in the vehicle,capturing an image of the field of interest with the camera, identifyinga region of interest in the image that is expected to convey vehicleinformation, and running an image processing model over the region ofinterest to extract vehicle information from the image.

According to another aspect of the disclosure, an example system forvehicle data collection is provided. The system includes a camera,positioned in a vehicle, to capture an image of a field of interest inthe vehicle, an image processing unit, at the vehicle and operativelycoupled to the camera, to run an image processing model to extractvehicle information from the image captured by the camera. The systemfurther includes a telematics system, at the vehicle and operativelycoupled to the image processing unit, to receive the vehicle informationdetermined by the image processing unit, and transmit the vehicleinformation to a vehicle data collection system.

The field of interest may include an information display. Theinformation display may include a dashboard of the vehicle, and theregion of interest may cover an instrument on the dashboard that conveysvehicle information. The instrument may be a turn signal indicator, afuel gauge, or another instrument type.

The field of interest may cover a vehicle safety feature, and thevehicle information extracted from the image may include whether thevehicle safety feature is engaged. The vehicle safety feature may be avehicle door.

Identifying the region of interest may involve locating the region ofinterest and determining a type of vehicle information that the regionof interest is expected to convey. The method/system may further involveselecting the image processing model, from a library of image processingmodels, that is appropriate to extract vehicle information from theregion of interest, based on the type of vehicle information that theregion of interest is expected to convey. The region of interest maycover a visual data source that conveys the vehicle information, themethod/system may further involve determining a format of the visualdata source, and selecting the image processing model is further basedon the format of the visual data source. The method/system may furtherinvolve controlling the camera to capture images in accordance with aset of image capture instructions associated with the image processingmodel. The image capture instructions may be to monitor a subfieldwithin the field of interest for a significant change, and to cause thecamera to capture an image of the subfield in response to detection of asignificant change.

The image may contain a plurality of regions of interest that are eachexpected to convey a type of vehicle information, and the method/systemmay further involve determining a layout of the plurality of regions ofinterest in the image, assigning an image processing model to each ofthe regions of interest, and controlling each of the image processingmodels to extract a respective type of vehicle information from itsassigned region of interest.

The method/system may further involve collecting additional vehicleinformation through a conventional telematics data pathway, determiningwhether a trigger condition in the vehicle information collected throughthe conventional telematics data pathway is satisfied, and in responseto satisfaction of the trigger condition, causing the camera to capturean image and causing the image processing model to extract vehicleinformation from the image.

The method/system may further involve determining whether a triggercondition in the vehicle information extracted by the image processingmodel is satisfied, and in response to satisfaction of the triggercondition, collecting additional vehicle information through aconventional telematics data pathway.

According to yet another aspect of the disclosure, an example method fortraining an image processing model for vehicle data collection isprovided. The method involves accessing an image of a field of interestin a vehicle captured by a camera in the vehicle, providing a userinterface to display the image, receive input that defines a region ofinterest in the image that is expected to convey vehicle information,and receive input that assigns a label to the region of interest thatassociates the region of interest with an image processing model that isto be trained to extract a type of vehicle information from the regionof interest. The method further involves contributing the image,labelled with the region of interest and the label associating theregion of interest to the image processing model, to a training datalibrary to train the image processing model.

According to yet another aspect of the disclosure, an example system fortraining an image processing model for vehicle data collection isprovided. The system includes a camera, positioned in a vehicle, tocapture an image of a field of interest in the vehicle, one or moreprocessors operatively coupled to the camera to access an image capturedfrom the camera, provide a user interface to display the image, receiveinput that defines a region of interest in the image that is expected toconvey vehicle information, and receive input that assigns a label tothe region of interest that associates the region of interest with animage processing model that is to be trained to extract a type ofvehicle information from the region of interest. The one or moreprocessors are further to contribute the image, labelled with the regionof interest and the label associating the region of interest to an imageprocessing model, to a training data library to train the imageprocessing model.

The method/system may further involve contributing the image, labelledwith the region of interest and the label associating the region ofinterest to the image processing model, to a library of training data totrain a localization model to locate similar regions of interest inother images.

The method/system may further involve contributing the image, labelledwith the region of interest and the label associating the region ofinterest to the image processing model, to a library of training data totrain a layout detector model to determine layouts of similar regions ofinterest in other images.

The image may include a plurality of regions of interest, wherein eachof the regions of interest cover a separate visual data source that isexpected to convey a different type of vehicle information. The userinterface may further be to receive input that indicates a type ofvehicle information expected to be conveyed by each visual data source,and the method/system may further involve contributing the image,labelled with each of the regions of interest and the labels associatingeach region of interest to a corresponding image processing model, to alibrary of training data to train a layout detector model to determinelayouts of similar regions of interest in other images.

The user interface may further be to receive input that indicates aformat type of each visual data source. Each of the visual data sourcesmay include an instrument on an instrumentation panel in the vehicle.The region of interest may cover a visual data source that conveys thevehicle information in accordance with a format type, the user interfacemay be to receive input that indicates the format type of the visualdata source, and the image processing model may be trained to extractvehicle information from an image of a visual data source of that formattype.

The field of interest may include an information display. Theinformation display may include a dashboard of the vehicle, and theimage may contain a region of interest that covers an instrument on thedashboard that conveys vehicle information. The field of interest maycover a vehicle safety feature, and the vehicle information extractedfrom the image may include whether the vehicle safety feature isengaged.

According to yet another aspect of the disclosure, examplenon-transitory machine-readable storage media comprising programminginstructions for vehicle data collection are provided. Exampleinstructions cause a processor to identify a region of interest in animage, captured by a camera located in a vehicle and pointed toward afield of interest in the vehicle, that is expected to convey vehicleinformation, and run an image processing model over the region ofinterest to extract vehicle information from the image.

According to yet another aspect of the disclosure, examplenon-transitory machine-readable storage media comprising programminginstructions for training an image processing model for vehicle datacollection are provided. Example instructions cause a processor toaccess an image of a field of interest in a vehicle captured by a camerain the vehicle, provide a user interface to display the image, receiveinput that defines a region of interest in the image that is expected toconvey vehicle information, and receive input that assigns a label tothe region of interest that associates the region of interest with animage processing model that is to be trained to extract a type ofvehicle information from the region of interest. The instructionsfurther cause the processor to contribute the image, labelled with theregion of interest and the label associating the region of interest tothe image processing model, to a training data library to train theimage processing model.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a schematic diagram of an example system for vehicle datacollection.

FIG. 2 is a schematic diagram of another example system for vehicle datacollection, detailing parallel pathways for conventional telematics datacollection and image analysis data collection.

FIG. 3 is a schematic diagram of another example system for vehicle datacollection, detailing the components of an image processing unit thatdetermines vehicle information from an image captured by a camera in avehicle.

FIG. 4 is a flowchart of an example method for vehicle data collection.

FIG. 5 is a schematic diagram of an example system for training an imageprocessing model for an image processing unit.

FIG. 6 is a schematic diagram of an example user interface for receivinguser feedback for training an image processing model.

FIG. 7 is a flowchart of an example method for training an imageprocessing model for an image processing unit.

DETAILED DESCRIPTION

Modern vehicles are typically equipped with a host of computer systemscomprising Electronic Control Units (ECUs) that control variousfunctions of the vehicle. These ECUs can communicate with one anotherusing a standardized communication protocol such as a Controller AreaNetwork (CAN) protocol, an automotive ethernet protocol, or anothermessaging protocol, to share data among one another. In many cases, themessages transmitted over such protocols conform to a standardizedhigher-layer protocol that defines how the data payloads contained inthe data messages are to be translated into vehicle information (e.g.,OBD2, SAE J1939, CANopen). These messages may be monitored (and in somecases actively requested) by a telematics system installed in thevehicle.

However, a growing number of vehicles, particularly electric vehicles(EVs), do not rely on any standardized higher-layer protocol, makingcertain kinds of vehicle information difficult to obtain. Even in thecase of internal combustion engine (ICE) vehicles, manufacturers areincreasingly using proprietary messages for specialized purposes,without publishing instructions for how such proprietary messages are tobe decoded. Thus, it is becoming increasingly difficult for telematicssystems to track certain kinds of vehicle information through theconventional approach of monitoring communication activity within avehicle.

Thus, the present disclosure describes systems and methods forcollecting vehicle data by analysing images of visual data sourceswithin a vehicle. A visual data source may include an informationdisplay, such as a dashboard (i.e., instrumentation panel), heads-updisplay, an indicator light, or any fixed location within the vehicle(e.g., a seatbelt, vehicle door, cargo area), that is expected to conveyvehicle information, and in particular, some sort of operating status ofthe vehicle. Machine learning models may be trained to extractinformation from images of these areas, and this information may becombined with conventionally-obtained telematics data to enhance and/orsupplement the information collected from the vehicle. Certain aspectsof image processing may be facilitated by a dedicated image processingunit with sufficient processing capability, located onboard the vehicle,thereby alleviating processing load from the telematics system itself.

FIG. 1 is a schematic diagram of an example system 100 for vehicle datacollection. The system 100 includes a vehicle 102 that is to be trackedby a telematics system. The vehicle 102 may be any type of vehicularasset, such as a passenger vehicle, transport truck, constructionequipment vehicle, sporting vehicle, utility vehicle, naval vessel,aircraft, or any other vehicular asset. For simplicity, a single vehicle102 is shown, but it is to be understood that the system 100 may includea plurality of vehicles to be tracked.

In other examples, the system 100 may include any type of non-vehicularasset to be tracked, such as a transport trailer, shipping container,pallet, shipped item, power generator, other machine, or any othernon-vehicular asset that is also tracked by a telematics system. In somecases, the asset may be associated with a vehicle (e.g., cargo onboardthe vehicle, trailer connected to a vehicle). Although the presentdisclosure is directed primarily to the case of tracking vehicles, theteachings described herein may be applied to the tracking ofnon-vehicular assets in certain circumstances.

Onboard the vehicle 102, the system 100 includes a telematics system 110that collects data from the vehicle 102. In some examples, thetelematics system 110 may be an asset tracking device coupled to thevehicle 102. In other examples, the telematics system 110 may beintegrated into the vehicle 102, either as an embedded device, orembodied in software and/or firmware operating on one or more computersystems onboard the vehicle 102 itself. Indeed, the telematics system110 may comprise any combination of the above possibilities. Thetelematics system 110 may be generally referred to as a telematicsdevice, asset tracking device, vehicle tracking device, telematicssystem, asset tracking system, vehicle tracking system, and the like.

Regardless of the configuration, the telematics system 110 comprises oneor more processors/controllers that obtain data from data sourcesassociated with the vehicle 102, which may include a sensor of thetelematics system 110 (e.g., accelerometer, GPS device), or a datasource of the vehicle 102 itself (e.g., an ECU communicating over a CANbus or automotive ethernet). The data collected by the telematics system110 may include the location of the vehicle 102 (e.g., obtained from aGPS device), motion data (e.g., obtained from an accelerometer), vehicledata such as vehicle speed, engine oil temperature, or proprietaryvehicle data (e.g., obtained from an ECU of the vehicle 102), or otherdata. Such data may be collected at the telematics system 110 inreal-time, near real-time, or in batches. Data collected directly from adata source may be referred to as raw data, which may pass through oneor more filtering/simplification steps at the telematics system 110before being transmitted to a server-side system. An example of afiltering step includes passing the raw data through a low-pass filterto remove noise that is not pertinent to the needs of the intendedpurpose of the data (e.g., a telematics service). Similarly, such datamay pass through a data simplification process, which may include a pathsimplification process (i.e., curve simplification process) such as theRamer-Douglas-Peucker algorithm or a variation thereof. The telematicssystem 110 then transmits/streams this data to a server-side system,such as the vehicle data collection system 120.

The vehicle data collection system 120 is remote from the vehicle 102and is in communication with the telematics system 110 via one or morecomputing networks and/or telecommunication networks (indicated asnetwork(s) 104), to receive and store the data collected by thetelematics system 110. The vehicle data collection system 120 includesstorage (e.g., one or more databases) to record data collected from thevehicle 102 (and other assets if applicable), including location data,trip/travel histories, sensor data, and other data. The vehicle datacollection system 120 may further store user accounts and other dataassociated with various assets and/or telematics systems for theprovision of telematics or analytics services. Thus, the vehicle datacollection system 120 generally includes one or more computing devices(e.g., servers, processors) to store the data and programminginstructions necessary to perform the functionality described herein,including at least one server with an appropriate network interface tocommunicate with the telematics system 110. The vehicle data collectionsystem 120 may include a plurality of systems/servers in a cloudcomputing environment. In addition to the functionality describedherein, the vehicle data collection system 120 may provide a telematicsservice, including live tracking, record keeping, and reporting servicesto end user (client) devices, and may further perform various analyticsservices, or forward the collected data to other systems for suchpurposes. In some contexts, the system 100 as a whole may be referred toas a telematics system, the vehicle data collection system 120 may bereferred to as a server-side telematics system (or as being at least apart thereof), and the telematics system 110 may be referred to as aclient-side telematics system.

Returning to the vehicle 102, the system 100 further includes a camera112 and an image processing unit 114. The camera 112 may include anyimaging device, such as a USB-operable camera, or any other imagingdevice capable of capturing an image within the vehicle 102. The camera112 may be capable of capturing still images and/or video imagery.

The image processing unit 114 may include one or any combination of aprocessor, microprocessor, microcontroller (MCU), central processingunit (CPU), processing core, state machine, logic gate array,application-specific integrated circuit (ASIC), field-programmable gatearray (FPGA), or similar, capable of executing, whether by software,hardware, firmware, or a combination of such, the actions performed bythe image processing unit 114 as described herein, including imageprocessing.

The camera 112 is onboard the vehicle 102, and is positioned to capturean image (or images) of a certain field of interest within the vehicle102 that contains a visual data source. As described in greater detailelsewhere in this disclosure, the camera 112 may be configured tocapture an image of an information display (e.g., dashboard or heads-updisplay) or another source of information that is pertinent to theoperation of the vehicle 102. Such images are transmitted to the imageprocessing unit 114, which extracts data from the image, and shares thatinformation with the telematics system 110. These images may be capturedin the form of still images or frames of a video. This data may be usedby the telematics system 110 (e.g., to trigger one or more events), orfor subsequent transmission to the vehicle data collection system 120.

The camera 112 may thereby be used to capture vehicle information thatcannot otherwise be obtained by the telematics system 110. For example,in cases where the telematics system 110 is unable to obtain vehicleodometer from the vehicle 102 directly (e.g., through a communicationport), then the camera 112 may capture an image of the odometer on theinstrumentation panel of the vehicle 102, and the odometer reading maybe extracted by the image processing unit 114. This data may be sharedwith telematics system 110 and transmitted to the vehicle datacollection system 120 as if it were collected by the telematics system110 itself, thereby supplementing the telematics system 110 with datathat it was unable to collect directly.

To this end, the image processing unit 114 may be loaded with a range ofimage processing models that are each trained to extract a certain typeof vehicle information from an image that contains a certain region ofinterest that covers a certain visual data source. For example, theimage processing unit 114 may be loaded with an image processing modelto extract an odometer reading from an image that includes an odometer,an image processing model to extract fuel level from an image thatincludes a fuel gauge, and so on.

Further, the image processing unit 114 may intelligently determine whatkinds of visual data sources are within view of the camera 112, selectone or more image processing models to extract vehicle information fromthose visual data sources, and run the appropriate image processingmodels accordingly. That is, the image processing unit 114 may analyse araw image captured by the camera 112, identify various regions ofinterest in the image (e.g., odometer, fuel gauge), and determine whichimage processing models are to be run against which regions of interest.

In some cases, the image processing unit 114 may control the camera 112to efficiently capture data from only those regions areas within itsfield of view that are determined to be regions of interest. Forexample, the image processing unit 114 may direct the camera 112 totransmit only a small portion of its field of view (i.e., a subfield),which corresponds to the location of a fuel gauge on an instrumentationpane, to the image processing unit 114 (i.e., a region of interest),thereby avoiding the needless capture and processing of image data thatdoes not convey vehicle information, thereby saving on bandwidth,processing resources, and enabling the camera 112 to capture images ofthe more important areas within view more frequently.

The system 100 further includes an image processing model trainingmodule 122, shown for example as being hosted by the vehicle datacollection system 120. The image processing model training module 122 isa place for users to submit training data that is used to train thevarious image processing models used by the image processing unit 114,as described in greater detail elsewhere in this disclosure. The vehicledata collection system 120 may also be used to evaluate the performanceof such image processing models. Once trained, the vehicle datacollection system 120 may push the necessary image processing models tothe image processing unit 114 (i.e., via the network(s) 104 andtelematics system 110), to be used in the operation of tracking thevehicle 102.

FIG. 2 is a schematic diagram of another example system 200 for vehicledata collection, detailing parallel pathways for conventional telematicsdata collection and image analysis data collection. The system 200 is toillustrate one example of how data collected from a vehicle, such as thevehicle 102 of the system 100 of FIG. 1 , may flow toward a vehicle datacollection system. Thus, various aspects of the system 200 may befurther understood with reference to the description of the system 100FIG. 1 and related systems.

The system 200 includes several vehicle ECUs 202, which may be sourcesof vehicle information for a vehicle, as described in FIG. 1 . Thevehicle ECUs 202 communicate over one or more Controller Area Network(CAN) buses 204. Under the conventional telematics data pathway, thevehicle ECUs 202 may make certain vehicle information available to acommunication port 206 (e.g., an OBD2 port) connected to the CAN bus(es)204, which may be accessed by a telematics system 208 (e.g., aself-contained tracking device that couples to the OBD2 port), andtransmitted to a vehicle data collection system.

However, certain types of vehicle information may not be readilyavailable to the telematics system 208 through the conventionaltelematics pathway. For example, the telematics system 208 may not beable to decode data messages that contain a certain type of vehicleinformation, or a certain type of vehicle information may not be madeavailable over the CAN bus(es) 204. For example, information about thefuel level of the vehicle may not be available. It nevertheless may bedesirable to collect this vehicle information. In such circumstances,the data that is unavailable over the conventional telematics datapathway may be collected under the image analysis data pathway.

Under the image analysis data pathway, the ECUs 202 may make certainvehicle information available to an information display 210 (e.g., adashboard, instrumentation panel or heads-up display) or another visualdata source. A camera 212 may capture an image of the informationdisplay 210, and an image processing unit 214 may process the image toextract vehicle information from the image. For example, the camera 212may capture an image of a dashboard of the vehicle that contains a fuelgauge, and the image processing unit 214 may identify the fuel gauge anddetermine a fuel level reading from the image. This data may be sharedwith the telematics system 208 and transmitted to the vehicle datacollection system. Thus, vehicle information that cannot be obtainedthrough the conventional telematics data pathway may be obtained throughthe image analysis data pathway.

In some examples, the data collected through the image analysis datapathway may be bundled into data packets that are indistinguishable, tothe vehicle data collection system, from data packets that contain datacollected through the conventional telematics data pathway. In otherwords, it may not be apparent to the vehicle data collection systemwhether the data was obtained through one pathway or the other. In otherexamples, data collected through the image analysis data pathway may bebundled into a data packet that includes metadata that identifies theinformation as being obtained through image analysis.

In some cases, where a certain type of information is available throughboth the conventional telematics data pathway and the image analysisdata pathway, both forms of information may be transmitted to thevehicle data collection system (with the respective data sources beingindicated), for the purposes of comparison. In some cases, the dataextracted through the image analysis data pathway may be transmittedalong with the source image from which the data was extracted, which,when combined with the same data collected through the conventionaltelematics data pathway at the same time, may be used to check theaccuracy of the image processing model used to extract that data, and totrain the image processing model accordingly. In some cases, thecollection of the same data from both data sources may be used merely toidentify a discrepancy, which may suggest an issue with the manner inwhich data is communicated through one pathway or the other.

Thus, vehicle information may be collected through an image analysisdata pathway, and additional vehicle information may be collectedthrough a conventional telematics data pathway. Although specificexamples are provided herein, it is to be appreciated that variations tothe system 200 are contemplated. For example, although the examplesystem 200 shown in FIG. 2 describes a case in which vehicle informationis transmitted over one or more CAN bus(es) 204, accessible by an assettracking device coupleable to the vehicle (telematics system 208), it isto be understood that the two parallel data collection pathways may besimilarly described in cases where a different communication protocol isin use (e.g., automotive ethernet), or when the telematics system 208 isintegrated into the vehicle itself (e.g., embodied in software).Further, the two parallel data collection pathways may be similarlydescribed in connection with obtaining vehicle information from any sortof visual data source (e.g., speedometer, odometer, warning light, turnsignal indicator), visible in any field of interest within the vehicle(e.g., vehicle dashboard, other field of interest).

FIG. 3 is a schematic diagram of another example system 300 for vehicledata collection, detailing the components of an image processing unit310 that determines vehicle information from an image captured by acamera in a vehicle. The image processing unit 310 may be understood tobe one example of the image processing unit 114 of FIG. 1 . Thus,various aspects of the system 300 may be further understood withreference to the description of system 100 of FIG. 1 and relatedsystems.

The system 300 includes a camera 302 (similar to the camera 112 of FIG.1 ), positioned in a vehicle, to capture images of a field of interestin the vehicle that contains a visual data source. In the example shown,the field of interest in the vehicle is presented as a vehicle dashboardthat contains various instruments (visual data sources) that conveyvehicle information, including a speedometer, tachometer, odometer, turnsignal indicators, fuel gauge, and engine oil temperature gauge. Thecamera 302 may be fixed (e.g., mounted) in a location in the vehiclethat is in view of the field of interest. For example, the camera 302may be strapped or mounted to the steering column, behind the steeringwheel, facing toward the instrumentation panel.

The system 300 further includes an image processing unit 310 (similar tothe image processing unit 114 of FIG. 1 ). The image processing unit 310is operatively coupled to the camera 302, for example, by any suitablewired or wireless connection. The image processing unit 310, asdescribed in greater detail below, receives images captured by thecamera 302 and analyses the images to determine vehicle information.That is, the image processing unit 310 may run one or more imageprocessing models to extract vehicle information from one or more imagescaptured by the camera 302.

The system 300 further includes a telematics system 350 (similar to thetelematics system 110 of FIG. 1 ). The telematics system 350 isoperatively coupled to the image processing unit 310, for example, byany suitable wired or wireless connection. The telematics system 350receives the vehicle information determined by the image processing unit310 and transmits the vehicle information to a vehicle data collectionsystem (similar to the vehicle data collection system 120 of FIG. 1 ).

The image processing unit 310 includes a camera interface 315,operatively coupled to the camera 302, to coordinate the capture ofimages by the camera 302. In some cases, the camera interface 315passively receives images captured by the camera 302. In other cases,the camera interface 315 actively controls the capture of images by thecamera 302. For example, the camera interface 315 may cause the camera302 to capture an image in response to an instruction received by theimage processing unit 310 from the telematics system 350 (e.g., based ona triggering event detected by the telematics system 350). As anotherexample, the camera interface 315 may cause the camera 302 to capture animage on a regular schedule on a continual/ongoing basis (e.g., every 1second, 30 seconds, or 60 seconds, depending on bandwidth capabilities).In still other examples, the camera interface 315 may cause the camera302 to capture an image only when a significant change in the field ofview of the camera 302 is detected (e.g., when an odometer increments,when a turn signal indicator light blinks, or when the camera field ofview or angle changes). Still further, the camera interface 315 maycause the camera 302 to actively monitor smaller portions of its fieldof view (certain regions of pixels), which are known to cover visualdata sources of interest, and to capture images of those regions, eitherperiodically or upon the satisfaction of certain trigger conditions.

The image processing unit 310 further includes a region of interestidentifier 320, operatively coupled to the camera interface 315, toidentify/locate one or more regions of interest in an image captured bythe camera 302 that is expected to convey vehicle information. That is,in the present example, the region of interest identifier 320 mayidentify areas of an image which correspond to a speedometer, odometer,fuel gauge, turn signal, or other areas. The region of interestidentifier 320 may identify such regions of interest by applying aplurality of localization models that are each trained to identify aregion of interest that corresponds to a particular visual data source.The application of one or more localization models may be referred to asthe application of a layout detector model. For example, the region ofinterest identifier 320 may apply one localization model to identifyodometer, one localization model to identify tachometer, and so forth.The region of interest identifier 320 determines not only an area wherea region of interest is located (e.g., a bounding box or pixelcoordinates), but also determines a label/identifier of the type ofvehicle information that a region of interest is expected to convey(e.g., odometer, fuel gauge). Such a label or identifier associates aregion of interest with an image processing model that is to be trainedto extract a type of vehicle information from the region of interest.Each region of interest may be identified by pixel coordinates of abounding box that contains the region of interest, the size anddimensions of such bounding boxes, or any other appropriate means todefine a region of interest in an image.

In the example shown in FIG. 3 , two regions of interest (R.O.I.) areidentified: one region of interest, labelled as R.O.I. 304-1, thatcorresponds to a speedometer, and a second region of interest 304-2,labelled as R.O.I. 304-2, that corresponds to an odometer.

The image processing unit 310 further includes an image processing modelcoordinator 330, operatively coupled to the camera interface 315 andregion of interest identifier 320, which selects one or more imageprocessing models, based on the identified regions of interest, toextract vehicle information from those regions of interest. The imageprocessing model coordinator 330 may attempt to match each identifiedregion of interest to the appropriate image processing model by matchinglabels of identified regions of interest to labels of image processingmodels. For example, where a region of interest was identified aspertaining to fuel gauge, the image processing model coordinator 330will select an image processing model that is trained to extract fuellevel from an image of a fuel gauge. Further, where a region of interestwas identified as pertaining to a specific format of fuel gauge, theimage processing model coordinator 330 may select an image processingmodel that is trained to extract fuel level from an image of a fuelgauge of that format.

The image processing model coordinator 330 may further configure thecamera 302, via the camera interface 315, to capture one or more imagesin accordance with a set of image capture instructions associated witheach region of interest identified to be in its field of view. Eachimage processing model may be associated with a “preferred” set ofinstructions for how the camera 302 is to capture images in a mannerthat is best suited for processing by the image processing model. Forexample, an image processing model may be associated with a set ofinstructions that requests that an image be captured in response to achange in the visual field of the camera (e.g., an image processingmodel for turn signal may be associated with a set of instructions forthe camera 302 to monitor a small area within its field of view (i.e.,subfield) corresponding to the turn signal indicator and capture animage each time there is a significant change therein (i.e., when theturn signal indicator blinks)). As another example, an image processingmodel may be associated with a set of instructions that requests that animage be captured periodically (e.g., an image processing model forspeedometer may be associated with a set of instructions that the camera302 is to capture an image every second, every thirty seconds, or everysixty seconds). As yet another example, an image processing model may beassociated with a set of instructions that requests that an image becaptured in response to a trigger condition being satisfied in anotherkind of data (e.g., an image processing model for seat belt indicatormay be associated with a set of instructions for the camera 302 tocapture an image when the vehicle shifts from park to drive asdetermined by the telematics system 350). Thus, the image processingmodel coordinator 330 may instruct the camera 302 to capture images inaccordance with the needs of the image processing models that areselected to capture information from the visual data sources that arewithin the field of view of the camera 302.

The image processing unit 310 stores a model library 340 in which arange of image processing models are stored, each image processing modelbeing trained to extract a certain type of vehicle information from animage that contains a certain region of interest. Each image processingmodel may employ one or more filtering, alignment, feature extraction,or other image processing techniques, that are specifically applicableto extract the kind of vehicle information it is designed to extract.For example, an image processing model that is configured to extract anodometer reading from an image of an odometer may apply one or morefilters, image alignment/registration, and optical characterrecognition, to extract an odometer reading from the image. As anotherexample, an image processing model that is configured to extract fuellevel from an image of a fuel gauge may apply one or more filters, and aprobabilistic model that estimates where the needle of the fuel gauge issituated between “full” and “empty”, to extract a fuel level readingfrom the image. Other image processing models may employ other imageanalysis techniques, such as neural networks, or other deep learningtechniques, to extract data from a variety of visual data sources. Anyof these image processing models may be selected by the image processingmodel coordinator 330 when needed, as described above.

An image filtering process may include the application of any imagefilter that improves the extraction of vehicle information from animage, such as motion blur reduction, glare reduction, a binary filter,image erosion, image sharpening, Otsu thresholding, or another imagefiltering technique. An image alignment process may include an imageregistration process in which one or more key points are extracted froma reference image (a previously captured image of the field of interest)and a source image (the image from which vehicle information is to beextracted), and the source image is aligned to the reference image byapplication of a transformation matrix. In other words, an amount ofrotation/translation or other shift/movement of the camera 302 betweenthe source image and the reference image is corrected.

Returning to the specific example shown in FIG. 3 , the image processingmodel coordinator 330 selects image processing model 342-1, which istrained to extract speedometer information from images of speedometers,and assigns this model to region of interest 304-1, and further selectsimage processing model 342-2, which is trained to extract odometerinformation from images of odometers, and assigns this model to regionof interest 304-2. The image processing model coordinator 330 may thencontrol each of the image processing models 342-1, 342-2 to extract arespective type of vehicle information from its assigned region ofinterest. The image processing model coordinator 330 may control eachimage processing model by feeding each respective image processing modelimage data (e.g., by feeding each image processing model image data thatcorresponds to its associated region of interest, or by feeding eachmodel an entire image), receiving the results, and when necessary,controlling the camera interface 315 to capture images in accordancewith the appropriate image capture instructions.

The model library 340 may further include one or more layout detectormodels which are configured to determine a layout of a plurality ofregions of interest in an image of a field of interest in the vehicle.As described above, the region of interest identifier 320 may apply alayout detection model, which may involve the application of one or morelocalization models to an image to identify regions of interest in theimage, to identify/locate regions of interest in an image. These modelsmay be stored in the model library 340 or another location accessible tothe region of interest identifier 320.

Although specific examples are provided herein, it is to be appreciatedthat variations to the system 300 are contemplated. For example, in somecases, the functionality of the image processing unit 310 and thetelematics system 350 may be performed by a single unit with sufficientmemory and processing capabilities. As another example, the system 300may be applied to extract vehicle information from other fields ofinterest, such as a heads-up display, another information display, oranother area within the vehicle that is expected to convey vehicleinformation, and other regions of interest may be identified, such asregions of interest that cover other instrumentation types. Further, themodel library 340 may contain different image processing models toextract vehicle information from different formats of the sameinstrumentation type (e.g., different formats of a fuel gauge). Theregion of interest identifier 320 may identify those different formattypes, and the image processing model coordinator 330 may selectappropriate image processing models for those formats.

FIG. 4 is a flowchart of an example method 400 for vehicle datacollection. The method 400 briefly describes one example of how thesystem 300 may collect vehicle information from an image, and thus forconvenience, the method 400 will be described with respect to theelements described in the system 300 of FIG. 3 . For greaterappreciation of the details and possible variations of the method 400,the description of the system 300 of FIG. 3 and related systems may bereferenced. Further, it should be noted that certain operations in themethod 400 may be embodied in programming instructions storable on anon-transitory machine-readable storage medium and executable by theimage processing unit 310 of FIG. 3 and/or related systems. However, itis to be understood that the method 400 may be performed by othersystems.

At operation 402, the camera 302 is positioned in a vehicle to bepointed toward a field of interest in the vehicle. At operation 404, thecamera 302 captures an image of the field of interest. At operation 406,the image processing unit 310 identifies a region of interest in theimage that is expected to convey vehicle information (e.g., via theregion of interest identifier 320). Prior to identification of a regionof interest, one or more filtering and alignment processes may beapplied to the image to allow vehicle information to be extracted fromthe image more reliably. At operation 408, the image processing unit 310runs an image processing model (e.g., image processing model 342-1,342-2, or another model in the model library 340) over the region ofinterest to extract vehicle information from the image (e.g., via imageprocessing model coordinator 330). The extracted vehicle information maybe transmitted to the telematics system 350, and thereby transmitted toa vehicle data collection system.

FIG. 5 is a schematic diagram of an example system 500 for trainingimage processing models for an image processing unit, such as the imageprocessing model training module 122 of the system 100 of FIG. 1 . Forgreater appreciation of the details and possible variations of thesystem 500, the description of the system 100 of FIG. 1 and relatedsystems may be referenced, but it is to be understood that the system500 is to be applicable to the training of other image processing units.

The system 500 includes a camera 502 (similar to the camera 112 of FIG.1 ), positioned in a vehicle, to capture an image of a field of interestin the vehicle. Images captured by the camera 502 may be accessed by theimage processing model training module 510 by any suitable means.

The system 500 further includes a user interface 520, provided by theimage processing model training module 510 or related system, to allowusers to assign labels to images captured by the camera 502 to be usedin the training of image processing models. Once labelled at the userinterface 520, the images may be added to a training data library 530,to be used to train image processing models.

The user interface 520 includes an image display component 512 thatdisplays an image captured by the camera 502. The image displaycomponent 512 may display a plurality of images, or may allow a user toselect one among a plurality of images for viewing, but a single imageis shown for simplicity.

The user interface 520 is configured to receive input that defines aregion of interest in an image that is expected to convey vehicleinformation. Such input may be received by, for example, a user drawinga box over the relevant portion of the image display component 512 thatcorresponds to a particular instrument on a vehicle dashboard that isexpected to convey vehicle information, by input of one or morecoordinates of the region of interest, or by another input means.

The user interface 520 is also configured to receive input that labels adefined region of interest with an information type that corresponds toan image processing model that is to be trained to extract vehicleinformation from the defined region of interest. Such input may bereceived by, for example, through a text box, drop down menu, radioselection, or other input means associated with a defined region ofinterest, in which a user inputs an identifier of a particular type ofvehicle information that is expected to be conveyed by the definedregion of interest. The user interface 520 may also include a vehicleinformation component 514 that is configured to receive user input thatdefines basic identifying information about the type of vehicle in whichthe image was captured, which may further be used to label the image,and used as part of the training data to train image processing models.

Thus, in the example shown, the image display component 512 may displayan image of a vehicle dashboard that includes instrumentation for aspeedometer, tachometer, odometer, fuel gauge, and engine oiltemperature gauge. A user of the user interface 520 may define a firstregion of interest by drawing a bounding box, indicated as R.O.I. input516, around the speedometer, and a second region of interest around thefuel gauge. A user may then label the defined region of interest withthe indicator “VehicleSpeed” in a text box, indicated as label input518, to indicate that it is expected that the defined region of interestwill convey information regarding the speed of the vehicle. The user maysimilarly label the region of interest defined around the fuel gaugewith a corresponding indicator. A user may also input information aboutthe vehicle in which the image was captured, such as the vehicle make,model, and year, into the vehicle information component 514. Oncelabelling is complete, a user may submit the labelled image (e.g., bypressing a “submit” button) to the image processing model trainingmodule 510, to be used for training of the appropriate image processingmodels.

An image that is labelled with one region of interest that contains onevisual data source may be used to train an image processing model thatis dedicated to extract vehicle information from that data source (e.g.,a particular instrument type on a particular vehicle type). An imagethat is labelled with more than one region of interest, each of whichcovers a separate visual data source that is expected to convey adifferent type of vehicle information (or in a different format), may beused to train multiple image processing models (e.g., an imageprocessing model that corresponds to each type of vehicle informationidentified in the image).

In some cases, the image processing model training module 510 maycompile a plurality of images captured within vehicles of the samevehicle type, that are labelled with the same type of region ofinterest, to train an image processing model to extract information fromthat type of region of interest in that vehicle. In other cases, theimage processing model training module 510 may compile a plurality ofimages captured within vehicles of different vehicle types, stilllabelled with the same type of regions of interest, to train imageprocessing models in a way that is agnostic as to the vehicle type fromwhich the images were collected. For example, the image processing modeltraining module 510 may use a plurality of images of speedometerscaptured from a plurality of different vehicle types (in whichinformation may be presented in different formats) in order to train animage processing model that is capable of identifying and readingodometer in different vehicle types.

Further, the image processing model training module 510 may put theimages captured herein toward training an image localization model, orlayout detector model, to determine a layout of plurality of regions ofinterest in an image captured in a vehicle of an unknown vehicle type.Thus, the image processing model training module 510 may contribute animage, with a labelled region of interest, to a library of training datato train a localization model to locate similar regions of interest inother images. Similarly, the image processing model training module 510may contribute the image, with the labelled region of interest, to alibrary of training data to train a layout detector model to determinelayouts of similar regions of interest in other images. A layoutdetector model trained in such a manner may thereby be configured todetermine, from an image captured from within an unknown vehicle type, alayout of various visual data sources within a field of interest (e.g.,a layout of instruments on an instrumentation panel). Such a layout maybe used to determine which image processing models should be applied towhich regions of an image.

The image processing model training module 510 may further train animage processing model using user feedback regarding predictions made byimage processing models. For example, the image processing modeltraining module 510, or a related system, may provide the user interface620, shown in FIG. 6 , for receiving user feedback regarding predictionsmade by image processing models. Such feedback may be used evaluate, andto further train the image processing models trained by the imageprocessing model training module 510. Such feedback may be aggregatedand analysed to evaluate the effectiveness of the image processingmodels.

The user interface 620 includes an image display component 612 thatdisplays an image that is expected to contain one or more regions ofinterest from which vehicle information may be extracted. Similar to theuser interface 520, the image display component 612 may display aplurality of images, or may allow a user to select one among a pluralityof images for viewing, but a single image is shown for simplicity.

The image display component 612 is configured to display a predictedregion of interest 616, generated by a layout detector model orlocalization model, around an area of the image that is predicted tocorrespond to a region of interest for a particular visual data source.Thus, in the example shown, the predicted region of interest 616 isshown around a speedometer of a vehicle dashboard. The user interface620 is configured to receive input that indicates whether the predictedregion of interest 616 is accurate. For example, the user interface 620may provide buttons 615 that allow a user to indicate whether thepredicted region of interest 616 accurately covers the speedometer,and/or may provide additional functionality to edit the predicted regionof interest 616, by any input means.

The user interface 620 further displays a predicted label 618 and apredicted value 619 corresponding to the information type, and theextracted vehicle information, determined by an image processing model.Thus, in the example shown, the predicted label 618 indicates that thespeedometer is to convey vehicle speed, and that the vehicle speedextracted is 108 km/h. As with the predicted region of interest 616, theuser interface 620 is configured to receive input that indicates whetherthe predicted label 618 and/or predicted value 619 are accurate, and/orprovide additional functionality to edit such information by any inputmeans (e.g., buttons 615).

The user interface 620 may also display predicted vehicle information614, determined by a layout detector model, that predicts the vehicletype of the vehicle in which the image was taken. Thus, in the exampleshown, the predicted vehicle information 614 indicates that the imagewas taken in a 2021 Ford F-150. As with the above, the user interface620 is configured to receive input that indicates whether the predictedvehicle information 614 is accurate, and/or provide additionalfunctionality to edit such information by any input means (e.g., buttons615).

Thus, a user may review the image displayed in the image displaycomponent 612, provide feedback as to the accuracy of the informationextracted from the image, and this feedback may be used to train imageprocessing models and/or layout detector models.

FIG. 7 is a flowchart of an example method 700 for training an imageprocessing model for an image processing unit. The method 700 brieflydescribes one example of how the system 500 and related systems may beused to train an image processing model, and thus for convenience, themethod 700 will be described with respect to the elements described inthe system 500 of FIG. 5 . For greater appreciation of the details andpossible variations of the method 700, the description of the system 500of FIG. 5 and related systems may be referenced. Further, it should benoted that certain operations in the method 700 may be embodied inprogramming instructions storable on a non-transitory machine-readablestorage medium and executable by the image processing model trainingmodule 510 of FIG. 5 . However, it is to be understood that the method700 may be performed by other systems.

At operation 702, the image processing model training module 510accesses an image of a field of interest in a vehicle captured by thecamera 502, which is located in the vehicle. At operation 704, the imageprocessing model training module 510 provides the user interface 520 todisplay the image. At operation 706, the user interface 520 receivesinput that labels the defined region of interest 516, which is a regionin the image that is expected to convey vehicle information. Atoperation 708, the user interface 520 receives input that labels thedefined region of interest 516 with a label that associates the definedregion of interest with a corresponding image processing model that isto be trained to extract a corresponding type of vehicle informationfrom the defined region of interest. At operation 710, the imageprocessing model training module 510 contributes the image, labelledwith the defined region of interest 516 and label that associates itwith a corresponding image processing model, to a training data library530 to train the image processing model. Such training data may furtherbe used to train image localization models or layout detector models.

Thus, vehicle information that may not be otherwise readily obtainedthrough a conventional telematics system may be obtained via imageanalysis of images taken of visual data sources within a vehicle.

Although specific examples are provided herein, it is to be appreciatedthat variations to the methods and systems described herein arecontemplated. For example, although the illustrations of visual datasources from which vehicle information may be extracted provided hereinhave been centred around common instruments present on most vehicledashboards, vehicle information may be extracted from other visual datasources.

For example, in cases where a camera is installed in an electricvehicle, certain types of vehicle information that are particular toelectric vehicles may be extracted from visual data sources on thevehicle dashboard which may or may not be accessible by other means,including a state-of-charge indicator that indicates whether the vehicleis in a charging state, a range indicator that indicates the remainingrange that the vehicle can travel on battery power, a battery healthindicator, or other visual data sources.

As another example, another visual data source may be present in anotherfield of interest in the vehicle other than the dashboard, such as, forexample, a heads-up display, a media display screen, or another type ofdisplay screen. A field of interest may include a storage area withinthe vehicle for storing cargo. In such examples, a camera may captureimages of the storage area, and an image processing unit may count anumber of boxes or otherwise quantify an amount of cargo stored in thestorage area, and relay that information to a telematics system. Anotherfield of interest may include an area of the vehicle associated with avehicle safety feature, such as a seatbelt, or a vehicle door, and acamera may monitor such areas to determine whether such safety featuresare engaged (e.g., whether a door is opened or closed, or whether aseatbelt is engaged or disengaged).

In some cases, the collection of vehicle information through theconventional telematics data pathway and the image analysis data pathwaymay be understood to be separate parallel processes that operateindependent of one another. In other cases, the collection of vehicledata through these different streams may impact one another and indeedenhance one another in various ways.

For example, the satisfaction of a certain trigger condition detected inthe conventional telematics pathway may trigger the camera to capture acertain type of information in a certain way through the image analysisdata pathway, in accordance with the mode of image capture necessary forthe appropriate image processing models. For example, a determination bya telematics system that a vehicle speed threshold has been exceeded maytrigger a camera to capture an image of a seatbelt indicator on thevehicle dashboard, an image of the driver side door to determine whetherthe door is open or closed, or another field of interest that contains avisual data source that may be of interest in the event that a vehiclespeed threshold is exceeded. As another example, a determination by atelematics system that a vehicle is making a turn, whether by GPS dataor accelerometer or another means, may trigger a camera to capture animage of a turn signal indicator to determine whether a turn signal wasengaged during a turn.

Or vice versa, as another example, the satisfaction of a certain triggercondition detected through the image analysis data pathway may triggerthe telematics device to capture a certain type of information in acertain way through the conventional telematics pathway. For example, adetermination through the image analysis data pathway that a seatbeltindicator indicates that a seatbelt has been disengaged may trigger thetelematics device to capture/record a GPS point that specificallyindicates the location at which the seatbelt was engaged. As anotherexample, the detection of the presence of a “check engine” warning lightthrough the image analysis data pathway may cause the collection ofvehicle diagnostic data through the telematics device at a higher ratethan usual so as to provide more detailed vehicle information forvehicle diagnostic purposes.

Moreover, as yet another example, the satisfaction of a certain triggercondition detected through the image analysis data pathway may triggerthe camera to capture a certain type of information in a certain waythrough the image analysis data pathway. For example, a determinationthrough the image analysis data pathway that a lane departure indicatorindicates that the vehicle is departing from a lane may trigger thecamera to monitor and capture imagery of a region of the vehicledashboard corresponding to a turn signal indicator to determine whetherthe vehicle had a turn signal engaged during the lane departure.

Thus, it should be seen that the teachings herein provide a number ofmethods and systems to collect vehicle information through the analysisof images captured from within a vehicle. It should be recognized thatfeatures and aspects of the various examples provided above can becombined into further examples that also fall within the scope of thepresent disclosure. The scope of the claims should not be limited by theabove examples but should be given the broadest interpretationconsistent with the description as a whole.

The invention claimed is:
 1. A method comprising: accessing an image ofa field of interest in a vehicle captured by a camera in the vehicle;providing a user interface to: display the image; receive input thatdefines a region of interest in the image that is expected to conveyvehicle information; and receive input that assigns a label to theregion of interest that associates the region of interest with an imageprocessing model that is to be trained to extract a type of vehicleinformation from the region of interest; and contributing the image,labelled with the region of interest and the label associating theregion of interest to the image processing model, to a training datalibrary to train the image processing model.
 2. The method of claim 1,further comprising: contributing the image, labelled with the region ofinterest and the label associating the region of interest to the imageprocessing model, to a library of training data to train a localizationmodel to locate similar regions of interest in other images.
 3. Themethod of claim 1, further comprising: contributing the image, labelledwith the region of interest and the label associating the region ofinterest to the image processing model, to a library of training data totrain a layout detector model to determine layouts of similar regions ofinterest in other images.
 4. The method of claim 1, wherein: the imageincludes a plurality of regions of interest, wherein each of the regionsof interest cover a separate visual data source that is expected toconvey a different type of vehicle information; the user interface isfurther to receive input that indicates a type of vehicle informationexpected to be conveyed by each visual data source; and the methodfurther comprises contributing the image, labelled with each of theregions of interest and the labels associating each region of interestto a corresponding image processing model, to a library of training datato train a layout detector model to determine layouts of similar regionsof interest in other images.
 5. The method of claim 4, wherein the userinterface is further to receive input that indicates a format type ofeach visual data source.
 6. The method of claim 4, wherein each of thevisual data sources comprise an instrument on an instrumentation panelin the vehicle.
 7. The method of claim 1, wherein the region of interestcovers a visual data source that conveys the vehicle information inaccordance with a format type, the user interface is to receive inputthat indicates the format type of the visual data source, and the imageprocessing model is to be trained to extract vehicle information from animage of a visual data source of that format type.
 8. The method ofclaim 1, wherein the field of interest comprises an information display.9. The method of claim 8, wherein the information display comprises adashboard of the vehicle, and the image contains a region of interestthat covers an instrument on the dashboard that conveys vehicleinformation.
 10. The method of claim 1, wherein the field of interestcomprises a vehicle safety feature, and wherein the vehicle informationextracted from the image comprises whether the vehicle safety feature isengaged.
 11. A system comprising: a camera, positioned in a vehicle, tocapture an image of a field of interest in the vehicle; one or moreprocessors operatively coupled to the camera to: access an imagecaptured from the camera; provide a user interface to: display theimage; receive input that defines a region of interest in the image thatis expected to convey vehicle information; and receive input thatassigns a label to the region of interest that associates the region ofinterest with an image processing model that is to be trained to extracta type of vehicle information from the region of interest; andcontribute the image, labelled with the region of interest and the labelassociating the region of interest to an image processing model, to atraining data library to train the image processing model.
 12. Thesystem of claim 11, wherein: the image includes a plurality of regionsof interest, wherein each of the regions of interest cover a separatevisual data source that is expected to convey a different type ofvehicle information; the user interface is further to receive input thatindicates a type of vehicle information expected to be conveyed by eachvisual data source; and the one or more processors are further tocontribute the image, labelled with each of the regions of interest andthe labels associating each region of interest to a corresponding imageprocessing model, to a library of training data to train a layoutdetector model to determine layouts of similar regions of interest inother images.
 13. The system of claim 12, wherein the user interface isfurther to receive input that indicates a format type of each visualdata source.
 14. The system of claim 12, wherein each of the visual datasources comprise an instrument on an instrumentation panel in thevehicle.
 15. The system of claim 11, wherein the region of interestcovers a visual data source that conveys the vehicle information inaccordance with a format type, the user interface is to receive inputthat indicates the format type of the visual data source, and the imageprocessing model is to be trained to extract vehicle information from animage of a visual data source of that format type.
 16. The system ofclaim 11, wherein the field of interest comprises an informationdisplay.
 17. The system of claim 16, wherein the information displaycomprises a dashboard of the vehicle, and the image contains a region ofinterest that covers an instrument on the dashboard that conveys vehicleinformation.
 18. The system of claim 11, wherein the field of interestcomprises a vehicle safety feature, and wherein the vehicle informationextracted from the image comprises whether the vehicle safety feature isengaged.
 19. A non-transitory machine-readable storage medium comprisingprogramming instructions that when executed cause a processor to: accessan image of a field of interest in a vehicle captured by a camera in thevehicle; provide a user interface to: display the image; receive inputthat defines a region of interest in the image that is expected toconvey vehicle information; and receive input that assigns a label tothe region of interest that associates the region of interest with animage processing model that is to be trained to extract a type ofvehicle information from the region of interest; and contribute theimage, labelled with the region of interest and the label associatingthe region of interest to the image processing model, to a training datalibrary to train the image processing model.